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Abstract. Numerous studies compared 
the effectiveness of various formats of 
video-based teaching, yet their focus has 
primarily been on relatively straightforward 
content, such as concepts and basic 
procedures. Research on the effectiveness 
of teaching complex content through 
different formats of videos remains limited. 
This study addresses this gap by conducting 
a well-controlled comparison between 
recorded video and narrated animation in 
the context of teaching physics problem- 
solving, a challenging content area with 
easily measurable difficulty levels. 

The study employed a controlled 
experimental design with a sample of 361 
upper secondary school students who had 
been randomly assigned to seven classes 
within a selected secondary school by the 
school administrator. Data were collected 
using pre- and post-test assessments 

that measured students’ problem-solving 
performance after video-based teaching. 
The results indicated that the effectiveness 
of recorded videos featuring the teacher's 
face was not significantly different from 
that of narrated animations that did not 
include the teacher's face, irrespective of 
the content’s difficulty level. These findings 
provide valuable insights for educators in 
selecting appropriate teaching formats 
for teaching challenging content through 
video-based education. They contribute to 
our understanding of teaching strategies 
and have practical implications for 
educators seeking to optimise teaching 
approaches in similar contexts. 
Keywords: physics problem-solving, 
secondary education, teaching/learning 
strategies, educational video 
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Introduction 


Education encompasses the methodologies and strategies employed 
by educators to facilitate learning, encompassing a range of activities de- 
signed to impart knowledge, skills, and attitudes. Video education, a subset 
of this broader educational practice, utilises video technology to deliver 
educational content, combining visual and auditory elements to enhance 
the learning experience. It is crucial for educators and educational design- 
ers to understand the effectiveness of different video formats in promoting 
student learning when they make decisions about what teaching formats 
to use. In recent years, especially due to the COVID-19 pandemic, the use of 
educational videos as a substitute for face-to-face teaching has gained sig- 
nificant attention in educational research (Brockfeld et al., 2018; Thilakumara 
et al., 2018;Wang et al., 2022). 

A wealth of studies have compared the effectiveness of teaching (stu- 
dent learning outcome due to teaching) across different formats, including 
asynchronous pre-recorded video teaching, synchronous online video teach- 
ing, and narrated animation, in teaching various subjects and content types 
(Christopoulos et al., 2023; Driscoll et al., 2012; Islam et al., 2020; Park et al., 
2018). One focus in this field has been the impact of showing the teacher's 
face on the effectiveness of video-based teaching. These previous studies 
have involved teaching mainly relatively easy content, such as scientific 
concepts and factual knowledge (Kizilcec et al., 2015; Van Wermeskerken & 
Van Gog, 2017; Wilson et al., 2018). There is a lack of studies exploring more 
complex teaching content with a high cognitive load. This study aims to ad- 
dress this gap by using physics problem-solving as teaching content, which 
is notably more challenging and cognitively demanding than the content 
previously studied (Byun & Lee, 2014; Park & Lee, 2004). Moreover, this study 
is the first to explore the interaction between content difficulty level and the 
comparison of learning outcomes when including and excluding the teacher's 
face in educational videos. This may provide very useful practical guidance 
to teachers for selecting the appropriate format of educational videos. 
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Theoretical Background 


Pre-recorded video refers to recordings of a teacher conducting face-to-face teaching in the classroom. This 
is also called ‘lecture capture’ ‘Narrated animation’ is used to refer to narrated visualisation (also called ‘voice-over 
PowerPoint’; Noetel et al., 2021). While pre-recorded video and narrated animation are collectively termed ‘educa- 
tional videos’in this study, they differ primarily in whether the teacher's face and/or body are shown in the recording. 

Social Agency Theory (SAT) posits that individuals actively shape their social environments and experiences 
through their choices and actions, rather than being passively shaped by external forces. Incorporating a teacher's 
presence in educational videos acts as a social signal that may trigger schemas of social interaction. As outlined by 
SAT, such social signals can provoke social reactions, such as heightened attention, leading to more thorough process- 
ing and improved learning transfer (Mayer, 2014). Moreover, according to the social-cue hypothesis (Colliot & Jamet, 
2018), these cues can also increase learners’ engagement and interest, which promotes better learning outcomes. 

Conversely, Cognitive Load Theory (CLT, Sweller, 1999), particularly in the context of multimedia learning 
(Mayer, 2014), recommends removing unnecessary elements from educational materials to avoid cognitive overload 
(Mayer & Fiorella, 2014). Humans tend to favour human faces (Gullberg & Holmqvist, 2006), with learners often 
focusing much of their attention on the teacher (Van Wermeskerken et al., 2018), thereby creating a visual rivalry 
with the educational content and splitting the learner's attention. This increases the extraneous cognitive load on 
learners, negatively impacting the learning process. 

In summary, theories suggest that while the appearance of a teacher's face in educational videos can act as 
a social prompt that boosts learning according to SAT, it can also be viewed as a source of unnecessary cognitive 
load that impedes learning according to CLT in multimedia learning. 


Literature Review 
Physics Problem-Solving 


Physics problem-solving requires a deep understanding of underlying principles and the ability to apply 
theoretical knowledge to real-world scenarios (Frensch & Funke, 2014; Park & Lee, 2004). Success in solving phys- 
ics problems demands many high-level skills, such as analytical skills, logical reasoning skills, creativity, and good 
mathematical calculation capability (Adams & Wieman, 2015). Previous studies have shown that students generally 
perceive physics problem-solving as a challenging task (Hegde & Meera, 2012; Mason & Singh, 2016). 

Compared to conceptual teaching and understanding, the teaching of problem-solving is relatively under- 
studied in the literature, especially for difficult physics problems. Yet, there are studies that have explored various 
aspects related to physics problem-solving, such as how students approach this task, expert—novice differences 
in problem-solving, factors influencing students’ problem-solving performance, and educational strategies aimed 
at developing expert-like problem-solving skills (Maries & Singh, 2018; Morphew et al., 2020; Price et al., 2021). 

Several studies (Bajracharya et al., 2019; De Leone & Gire, 2006; Justice et al., 2022; Kohl & Finkelstein, 2005) 
have compared the effectiveness of different educational approaches in developing students’ problem-solving 
skills in physics. Selcuk and Calyskan (2008) demonstrated that teaching problem-solving strategies improved stu- 
dents’ physics problem-solving. Similarly, Caliskan et al. (2010) analysed the effects of a problem-solving teaching 
strategy and confirmed its positive impact on students’ physics achievement, attitude toward physics, self-efficacy, 
and problem-solving skills. 

Advancements in technology have facilitated the development of computer-based problem-solving coaches 
and tools that employ interactive methods to assist students in becoming adept problem-solvers (Marshman et 
al., 2020; Ryan et al., 2016; Tse et al., 2019). This progression is endorsed by Koenig et al. (2022), who argued that 
interactive online tutorials contribute to the enhancement of students’ physics problem-solving skills. 

In summary, the findings from these studies serve to underscore the formidable challenges inherent in the 
domain of physics problem-solving. They shed light on the intricate and multifaceted nature of the discipline, where 
students must navigate complex theoretical frameworks and apply them to real-world scenarios with precision 
and ingenuity. Moreover, these studies underscore the importance of imparting robust physics problem-solving 
skills within the broader domain of science education. By equipping students with the ability to analyse, reason 
logically, and creatively tackle physics problems, educators empower them to become critical thinkers and problem 
solvers in various scientific contexts. In essence, these studies collectively underscore the pivotal role of physics 
problem-solving in science education and the growing research interest in this topic. 
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The Effect of Showing the Teacher's Face in Educational Videos 


Common educational video types include recorded video of classroom lectures (also called ‘lecture capture’; 
Danielson et al.,2014;Toppin, 2011; Wiese & Newton, 2013; Witton, 2017), voice-over PowerPoint (also called ‘voice- 
over presentation’ or ‘narrated animation’; Griffin et al., 2009; Noetel et al., 2021), and Khan Academy-style video 
lectures (Chorianopoulos & Giannakos, 2013), all of which can be either synchronous (live-streamed) or asynchro- 
nous (pre-recorded). Recently, videos including a virtual teacher have gained popularity (Horovitz & Mayer, 2021; 
Lawson et al., 2021). One focus in comparisons of the effectiveness of different types of video teaching format is 
determining whether including the teacher's face in the video boosts student learning. 

Previous research has suggested that the presence of the teacher's face in the video has two competing ef- 
fects. One effect is to increase learners’ attention to a procedure (e.g., drawing a diagram, writing down equations 
one-by-one) while the teacher explains, which facilitates learning (Kizilcec et al., 2015; Wilson et al., 2018). The 
other effect is a learning cost. Learners’ attention may be drawn to irrelevant visual characteristics of the teacher, 
potentially distracting them from the conceptual content of the presentation (Wilson et al., 2018). Previous studies 
have found that these two competing effects usually cancel each other out and that incorporating the teacher's 
image in an onscreen lesson does not yield significant positive effects on learning (Kizilcec et al., 2015; Van Wer- 
meskerken & Van Gog, 2017; Wilson et al., 2018). This is likely due to the limitations of cognitive processing in 
working memory (Mayer & Fiorella, 2014). When learners focus on the teacher, they divert their attention from the 
visually relevant teaching content, and if they are preoccupied with the teacher's mouth movements, they might 
miss the message's core content. Essentially, including the teacher's image on the screen may lead to extraneous 
processing — a cognitive activity that does not align with the desired learning outcomes and thus wastes valuable 
cognitive processing capacity (Mayer & Fiorella, 2014). 

Research on teacher presence has been advanced by manipulating distinct aspects of the teacher's gaze, 
gestures, voice, or presentation method (e.g., Fiorella & Mayer, 2018; Ouwehand et al., 2015; Pi et al., 2022). For ex- 
ample, Van Wemeskerken et al. (2018) used eye-tracking to observe how teacher presence affected students’ visual 
focus during a lesson. They found that students’ focus on the teacher remained consistent throughout the lesson, 
suggesting that human faces hold a distinct allure as social cues. The teacher's presence, as conveyed through their 
face, might capture and sustain students’ attention without being distracting enough to hinder learning. Fiorella 
and Mayer (2018) suggested that the lack of observed differences in learning outcomes across these studies might 
be related to the content's complexity. For instance, teacher presence may be beneficial for easy tasks, have no 
effect on moderate tasks, and impair performance for very complex tasks. Yet, there have been only limited studies 
offering empirical evidence on the impact of content (learning task) difficulty level. 


Research Questions 


The current body of evidence suggests that the inclusion of the teacher’s face in video teaching does not 
significantly impact the effectiveness of education for simpler content. However, there appears to be a lack of 
empirical research examining how the difficulty level of content affects the comparative effectiveness of different 
teaching formats. To the author's knowledge, no studies have investigated whether the observed lack of difference 
in effectiveness persists across varying levels of problem difficulty. It remains uncertain whether this lack of differ- 
ence is limited to simpler problems or if the disparity in teaching effectiveness between the formats becomes more 
pronounced as content complexity increases. By comparing the effectiveness of teaching physics problem-solving 
through pre-recorded video and narrated animation, this study seeks to answer the following research questions: 

RQ1: Does the effectiveness of physics problem-solving teaching differ across pre-recorded video that 
shows the teacher's face and narrated animation that does not show the teacher's face? 

RQ2: How does the difference in effectiveness between pre-recorded video and narrated animation relate 
to the difficulty level of the problem? 


Research Methodology 
Background 


This study compares the effectiveness of teaching physics problem solving using two different video formats: 
recorded video and narrated animation. Employing a quantitative research approach, the study was designed as 
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a randomized controlled trial to ensure the rigour and reliability of findings. Our theoretical framework draws on 
cognitive load theory, social agency theory and multimedia learning principles, positing that different teaching 
formats can variably affect students’ cognitive load, social cue, and attention allocation. By comparing the learning 
outcomes of pre-recorded videos and narrated animations, this research seeks to contribute to the understanding 
of the optimal video teaching format for challenging content. The scope of the study is focused on the teaching 
effectiveness of these video formats as they pertain to complex physics problems, thereby offering insights that 
could guide future educational practices. 


Sample Selection 


This study was conducted during the first semester of a two-year-long elementary physics course at a large public 
upper secondary school. The course, which is non-calculus-based, covers classical mechanics, electromagnetism, optics, 
and thermal physics, thereby preparing students for further studies in STEM subjects at university. Each of the 10" 
grade’s 26 classes contains between 50 and 55 students. Students in a particular class remain in the same classroom 
for all compulsory learning activities, including lectures, discussions, and lab work. There is only one 40-minute learn- 
ing session of a non-STEM subject (e.g., economics, social science) each week that is elective and therefore different 
for each individual student. All 10"-grade students participate in the same science course using identical course 
materials. In the semester during which the experiment was conducted, the STEM subjects included mathematics, 
physics, chemistry, biology, information technology, and engineering. Each week, the students receive three forty- 
minute teaching sessions in physics and are expected to spend about 3 hours related to homework assignments. 

Upon enrolment, which occurred three months prior to the experiment, students were randomly assigned to 
their respective classes by the school administration. From the 26 classes in the 10" grade, seven were randomly 
selected for this study. Out of the 361 students in the seven classes, 361 consented to participate in the study. The 
assignment to either the pre-recorded video group or the narrated animation group was also randomised at the 
class level. Specifically, a random selection process determined that three classes would be assigned to the video 
group and the remaining four classes to the animation group, ensuring a random distribution of teaching formats 
among the students. In the end, 0 students dropped out, which resulted in a final sample of 361 students—com- 
prising 180 male and 181 female students—all aged between 15 and 16 years. 

All students completed the school’s midterm physics exam three weeks before the study commenced. The 
scores from this exam were used to assess the students’ physics abilities and confirm the homogeneity of the 
sample across the groups. 


Instrument and Procedures 


Physics problem-solving usually imposes a high cognitive load and requires higher-order scientific reasoning 
and sustained focus, making it challenging for learners (Byun & Lee, 2014; Frensch & Funke, 2014; Park & Lee, 2004). 
The difficulty level of a problem and teaching effectiveness can be measured directly and reliably using a pre-test 
and a post-test with similar problems. These features make physics problem-solving a suitable representation of 
challenging content for this study, which examines whether the difficulty level of the teaching content impacts 
the effectiveness of two commonly used forms of video teaching. 

Numerous tools exist for measuring physics learning. These predominantly focus on conceptual understanding 
and attitudes towards the subject (Brundage & Singh, 2023; Gurler & Baykara, 2020; Hestenes et al., 1992; Marake et 
al., 2022; Maloney et al., 2001). However, specialised instruments for measuring complex physics problem-solving 
abilities are less common (Adams, 2015). Given their intricate nature and variable difficulty levels, problems selected 
from national college entrance exams are utilised for teaching and assessment in this study. 

This study employed four physics problems of differing difficulty levels, corresponding to the uni-structural, 
multi-structural, relational, and extended abstract categories according to the Structure of Observed Learning 
Outcomes (SOLO) taxonomy (Biggs & Collis, 2014; Brabrand & Dahl, 2009; Yildiz Durak & Atman Uslu, 2023), as 
shown in Table 1. The problems’ difficulty levels were also evaluated using expert ratings. A team of experienced 
physics educators, including two expert-level upper secondary school teachers and three university professors 
specialising in difficulty level, rated the problems on a 1-to-5 scale. The average of these ratings determined the 
level of difficulty. The results from both the expert ratings and the SOLO taxonomy categorisation are consistent 
and are presented in Table 1. The problems are ranked from lowest to highest difficulty as Problem 1 (easy), Problem 
2 (medium), Problem 3 (difficult), and Problem 4 (very difficult). 


573 


https://doi.org/10.33225/jbse/24.23.570 


Journal of Baltic Science Education, Vol. 23, No. 3, 2024 


RECORDED VIDEO VERSUS NARRATED ANIMATION IN TEACHING PHYSICS PROBLEM- ISSN 1648-3898 


/Print/ 

SOLVING: THE INFLUENCE OF PROBLEM DIFFICULTY LEVEL 
plang iahin lie ISSN 2538-7138 /oniine/ 
Table 1 
Difficulty Level of the Problems 

Problem SOLO level Description Problem difficulty Difficulty rating 

number categorisation by experts 

Pre-structural Questions typically involve basic knowledge or concepts. 
Problem 4 ivaicetnichical Questions involve the use of a single formula or principle Easy 24 
to solve a problem. 
Problem 2 Multi-structural Questions involve comprehensive application of multiple Nediumn 34 


physics concepts or principles. 


Questions require students to organically combine 
Problem 3 Relational different concepts, formulae, and principles to devise a Difficult 4.6 
complete problem-solving approach or solution. 


Problem 4 Extended Questions involve in-depth exploration and innovative Very difficult 5.0 
Abstract application of physics phenomena and principles. 


Originally, the problems appeared as multiple-choice questions on national college entrance exam papers, 
with one out of four choices being correct. To minimise the possibility of students guessing the correct answer, 
Problems 1 and 4 were converted into short-answer questions. The answers to Problems 2 and 3 are not numbers, 
so converting them into open-ended short-answer questions would make these questions very time-consuming 
to grade. They were converted to eight-item multiple-choice questions featuring two correct items. Only students 
who selected the two correct items received a ‘correct’ score for these problems. Since each item can either be se- 
lected or not selected, there are 2° possible response combinations, which is equal to 256. The chance of randomly 
guessing the correct answer for Problems 2 and 3 is thus only 1 in 256. Performance on each problem was rated 
using a binary scale. The student scored 1 if they answered the problem correctly and 0 otherwise. This meticulous 
conversion process, coupled with the binary scoring system, ensured a high level of objectivity in scoring, thereby 
enhancing the reliability of our assessment method. 

These four problems are referred to as the ‘pre-test problems’ They were used to filter out the students who 
already knew how to solve these kinds of problems. These students are either so intelligent that they can figure 
out how to tackle such a problem on their own the first time they encounter it, or they have followed a private 
academy (referred to as ‘shadow education’) learning programme outside of school that has taught them how to 
solve such problems already (Byun & Lee, 2014). The post-test data of only those students who did not answer the 
corresponding pre-test problem correctly were extracted and analysed. This approach ensures that the effective- 
ness of each video format is in transforming students from not knowing to knowing was compared. One possible 
scenario would be where a pre-recorded video about how to solve Problem 1 is delivered to 100 students and, 
ultimately, 80 students solve the post-test problem named Problem 1 Post. Without a pre-test, one cannot assert 
that the teaching contained in the videos effectively transformed 80 students from not knowing how to solve this 
type of problem to knowing how to solve it. This is because, possibly, 15 out of the 80 students who solved Problem 
1 Post did so due to other reasons (e.g., shadow education, innate intelligence) rather than the video teaching. 
The Pre-test Problem 1 serves to filter out those 15 students and ensure that learning (measured by the successful 
solution of the post-test problem) is attributed to the video teaching. 

Four problems, referred to as ‘Problem 1 Post’ ‘Problem 2 Post; ‘Problem 3 Post; and ‘Problem 4 Post; were 
chosen as post-test problems to measure the students’ performance after they had been taught on how to solve 
each corresponding pre-test problem. The post-test problems were selected based on the criterion that solving 
them required the same techniques and strategies used to solve the corresponding pre-test problems. For instance, 
both Problem 1 and Problem 1 Post are based on the equilibrium condition and use the same techniques of draw- 
ing a free body diagram, decomposing the forces into two perpendicular directions, and writing down and solving 
the force equation for two orthogonal directions to obtain the solution. Problem 1 Post and Problem 2 Post were 
selected from past papers of the National College Entrance Exam, and Problem 3 Post and Problem 4 Post were 
chosen from regional mock college entrance exams. All four post-test problems can be found in the appendix. 
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Performance on the post-test for each problem was based on a binary outcome: either correct or incorrect. The 
post-test problems were designed in the same way as the pre-test problems to minimise the possibility of students 
guessing the correct answer. 

Figure 1 illustrates the experimental procedure of the study and also constitutes an independent experiment. 
Students (N=361) participated in a midterm physics exam which tested their existing physics problem-solving 
skills and knowledge. Their scores on the exam were used as indicators of their existing level of understanding in 
physics to check the homogeneity between the experimental and control groups. The participants were then all 
asked to take a pre-test consisting of the four problems, with difficulty levels of easy, medium, difficult, and very 
difficult (called ‘Problem 1; ‘Problem 2/ ‘Problem 3; and ‘Problem 4; respectively). The pre-test was timed, but the 
students were allowed sufficient time (25 min in total) to attempt all the problems. 


Figure 1 
Experimental Procedure 


Mid-term physics exam 
(N=361,7 classes) 
Pre-test with Problem 1, Problem 2,Problem 3 and Problem 4 
(N=361,7 classes) 


Recorded Video Narrated Animation 
(N=154, 3 classes) (N=207, 4 classes) 


Post-test with Problem 1 Post, Problem 2 Post, Problem 3 Post and Problem 4 Post 
(N=361,7 classes) 


Teaching on how to solve the problem was delivered to the students, with the video group receiving pre- 
recorded video teaching and the animation group narrated animation. As an example, Figure 2 illustrates the teacher 
teaching how to solve Problem 4 in the two different video formats. The teaching on how to solve each problem 
was written as scripts. For each problem, the teacher followed the script to showcase the problem and illustrate the 
underlying physics in both teaching formats. The duration of the teaching on how to solve each problem is given 
in Table 2. As shown in Table 2, the teaching time allocated for each problem was kept the same for both formats. 


Table 2 
Duration of Teaching on How to Solve Each Problem in Each Mode 


Problem number Recorded video Narrated animation 
Problem 1 04min 27sec 04min 27sec 
Problem 2 04min 33sec 04min 33sec 
Problem 3 05min 48sec 05min 48sec 
Problem 4 04min 55sec 04min 55sec 


Both video formats illustrating the process of how to solve each problem used the same teacher and narra- 
tion. The teacher has 15 years of teaching experience in two upper secondary schools. He holds a master's degree 
in physics and a certificate in teaching upper secondary school physics. His profile is representative of that of a 


575 


https://doi.org/10.33225/jbse/24.23.570 


Journal of Baltic Science Education, Vol. 23, No. 3, 2024 


RECORDED VIDEO VERSUS NARRATED ANIMATION IN TEACHING PHYSICS PROBLEM- ISSN 1648-3898 | jerinty 
SOLVING: THE INFLUENCE OF PROBLEM DIFFIGULTY LEVEL 
pales ISSN 2538-7138 soniiney 


typical upper secondary school physics teacher. The teaching content of the recorded video was a recording of a 
real teaching session, without any modifications. The recorded videos were played by another teacher to the whole 
class in a classroom; therefore, all students watched the videos together without being able to pause, skip, fast 
forward, or repeat. These measures ensure that all other variables were well controlled and that the comparison 
is only between the teaching formats. The voice-over PowerPoint-based animation was designed and produced 
by the same teacher using his own audio track from the recorded video. The visual changes in the animations 
followed the timing of the audio track from the recorded videos, and the same illustrations were used in both 
formats wherever possible. Thus, the absence versus the presence of the teacher's face/body was the sole point 
of difference between the two formats. 


Figure 2 
The Two Formats of Video on How to Solve Problem 4. (a) Screenshot of the Pre-Recorded Video; (b) Screenshot of the Narrated 
Animation. 
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(b) 


After the lesson, a post-test was administered to all students in both groups, in which they were asked to solve 
a problem that was very similar to Problem 1 (i.e., Problem 1 Post) to determine how well they could transfer what 
they had learned to solve a similar problem. Solving Problem 1 Post requires using the same physics principles, 
problem-solving techniques, and scientific reasoning used to solve Problem 1, which was taught in the lesson before 
the post-test. As in the pre-test, the post-test problems had only binary outcomes: either correct or incorrect. The 
percentage of each group that gave the correct answer was used to rate the teaching effectiveness. 

The above process was repeated for Problems 2, 3, and 4. 
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Data Analysis Procedure 


In order to avoid disturbing the routine teaching of the classes participating in the experiment, we adopted 
a simple pre-test—post-test approach in the experiment, with all students grouped into different formats of video 
teaching by class. Therefore, it was not necessary to assign students to different groups for each problem. However, 
students were originally randomly assigned to the classes, and the classes were randomly selected from the school, 
so the participants still constitute a purely random sample of the school. 

The data analysis procedure was designed to ensure reliable results. A detailed figure illustrating this process 
is included in the appendix. Taking Problem 1 as an example, all students completed Problem 1 in the pre-test, 
but only those who failed to solve the problem in the pre-test entered the next stage of data analysis. For each 
group, firstly the distribution of the midterm physics exam scores was examined. Secondly, the invariance among 
the midterm physics exam scores of the two groups was verified, ensuring that there were no statistically signifi- 
cant differences between groups. Lastly, the percentage of the group with a correct answer to Problem 1 Post was 
calculated as an indicator of the teaching effectiveness of the format to which this group was exposed. 


Research Results 
Invariance of the Sample Groups 


Table 3 presents the results of the pre-test. The participants answered each pre-test problem either correctly or 
incorrectly. Among the 361 initial participants, 125, 194, 310, and 349 answered Problems 1, 2, 3, and 4 incorrectly, 
respectively, indicating well-differentiated levels of difficulty for these problems. This result is consistent with the 
categorisation of the problems according to the difficulty level given in Table 1. 


Table 3 
Results of the Pre-Test 


Number of initial Number of students 


Problem difficulty level Number of students who solved the problem 


participants who failed to solve the problem 
Easy 361 236 125 
Medium 361 167 194 
Difficult 361 51 310 
Very difficult 361 12 349 


Table 4 
Descriptive Results of the Midterm Physics Exam Scores 


Problem difficulty level Format xX n SD xX 
Recorded video 67.73 59 12.59 70 
Easy 

Narrated animation 67.89 66 12.79 70 
Recorded video 69.31 87 11.34 73 

Medium 
Narrated animation 67.85 107 12.47 72 
Recorded video 71.68 142 11.99 74 

Difficult 
Narrated animation 72.56 168 10.79 74 
Recorded video 71.73 146 11.86 74 

Very difficult 

Narrated animation 71.91 203 11.99 73 


Note. X= mean;n = number of students; SD = standard deviation; X= median. 


Four t-tests were conducted on the midterm physics exam scores to verify the absence of statistically sig- 
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nificant differences in participants’ prior physics knowledge and physics problem-solving skills, with the results 
shown in Figure 3. 

As indicated in Figure 3, no statistically significant differences were observed between the two groups’ midterm 
physics exam scores for any of the four problems. The t-test results indicate good invariance in the midterm phys- 
ics exam scores, which are used as an indicator of the students’ physics ability. Not only are the mean values very 
similar; all the p-values are well above .05 (p, = .94, pz = .40, p3 = .50, py = .89). In addition, the distributions 
of the midterm physics exam scores for the different groups are very similar, as shown in Figure 3. 


Figure 3 
The Distribution of Midterm Physics Exam Scores Between the Two Groups 
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Performance on Post-Test Problems 


Table 5 presents the post-test results for all four problems. It can be seen that the general trend is a decline in 
the percentage of correct answers (effectiveness of teaching) from Problem 1 to Problem 4, which again supports 
our categorisation of the problems’ difficulty levels. Fig. 4 presents the failure rates for all four problems. Three chi- 
square tests were conducted to compare the failure rates between any two adjacent problems. All of these tests 
revealed statistically significant differences (p < .001 for all three comparisons). 
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Post-Test Results for the Two Groups 
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Effectiveness of 


problem level Ponnet wne Taledte solve the answers in the post-test teaching, % 
pre-test problem 
Recorded video 59 44 75 
Problem 1 Easy 
Narrated animation 66 50 76 
Recorded video 87 4 47 
Problem 2 Medium 
Narrated animation 107 50 47 
Recorded video 142 31 22 
Problem 3 Difficult 
Narrated animation 168 52 31 
Recorded video 146 10 re 
Problem 4 Very difficult 
Narrated animation 203 17 8 
Figure 4 


Failure Rate (Percentage of Incorrect Answers in the Population) in Student Attempts to Solve the Post-Test Problems. 
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Note. ns=no significant difference; ***=significant difference with p < .001. 


Comparing Teaching Effectiveness of Recorded Video and Narrated Animation 


Difficult 
Problem 


Very Difficult 
Problem 


Figure 5 compares the effectiveness of teaching physics problem-solving using recorded video and narrated 
animation. 
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Figure 5 
Percentage of Correct Solutions to the Post-Test Problems Among Students Exposed to Video and Narrated Animation for All 
Four Problems. 


Post-test 
Wy 
on 


Video Animation Video Animation Video Animation Video Animation 
Easy Problem Medium Problem [II Difficult Problem ES Very Difficult Problem 


Note. ns=no significant difference; *=significant difference. 


It is interesting to note that regardless of the difficulty level of the problem, there are no statistically significant 
differences between the effectiveness of video and narrated animation, as evidenced by the results of chi-square 
tests (p-values . 88, . 96, . 71, and . 60, respectively). The animation format in this study is a voice-over PowerPoint 
that uses the same audio used in the pre-recorded video lesson. 


Discussion 
Implications of the Results 


The core finding of this study is that there is no statistically significant difference in the effectiveness of teaching 
physics problem-solving using pre-recorded videos (in which the teacher's face is visible) and narrated animations 
(in which the teacher's face is not visible) across various levels of problem difficulty. The research design rules out 
the influence of other variables, with the main difference between the recorded video and the narrated animation 
being whether the teacher's face/body is shown in the presentation. 

The inclusion of the teacher's face in the video presentation may increase learners’ attention to the teacher's 
explanations, which would facilitate learning. On the other hand, the teacher’s presence may draw learners’ at- 
tention to irrelevant visual characteristics of the teacher, potentially distracting them from the teaching content 
(Wilson et al., 2018). When learners switch their attention between the teacher and the content, learning could be 
negatively influenced, a phenomenon known as the split-attention effect (Ayres & Sweller, 2014). This is because 
it imposes an additional cognitive load on the learner, according to cognitive load theory. Previous studies have 
found that these two competing effects often cancel each other out, leading to no difference in students’ learning 
outcomes (Van Wermeskerken & Van Gog, 2017; Wilson et al., 2018). For example, Kizilcec et al. (2015) found that 
showing the teacher's face in a video lesson about sociology had no effect on the learning outcomes of adult learn- 
ers. Wermeskerken and Van Gog (2017) investigated whether the visibility of the teacher's face and the provision 
of gaze guidance in a video - specifically comparing videos in which the face was not visible, was visible with gaze 
guidance, or was visible without gaze guidance (i.e., when the teacher stared straight into the camera) — affected 
learning outcomes while college students majoring in psychology were being taught procedural knowledge in 
chemistry. Wilson et al. (2018) did not find a meaningful difference in learning performance following exposure 
to different video types (i.e. podcast, audio with text, or audio with visuals of the teacher). Recently, Sondermann 
and Merkt (2023) showed that the inclusion of the teacher as a‘talking head’ in educational videos teaching adults 
factual knowledge did not affect learning outcomes, regardless of the slide type and presentation type of the videos. 

Consistent with these studies, the findings of the present study confirm a similar pattern in the context of 
complex physics problem-solving. However, unlike factual knowledge, scientific concepts, and simple procedures, 
physics problem-solving involves an inherently complex process of reasoning, high cognitive load, and analytical 
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skills. The result of the present study suggests that although learning complex physics problem-solving is a high 
cognitive demand process, it does not alter the balance between the social cue benefit and the cognitive load 
increase disadvantage of including the teacher's face in educational videos. A significant practical implication is 
that teachers can employ either recorded videos or narrated animations without affecting the effectiveness of 
teaching, even for difficult and cognitively demanding content such as physics problem-solving. 


Quality Control 


The study design featured several controls to ensure consistency and avoid bias. These were as follows: 

(i) The variables being compared were well controlled. The video version of the teaching was a recording of the 
face-to-face teaching, using the same teacher and exactly the same content. The recorded videos were played by a 
teacher (not the one in the video) to the whole class together; therefore, all students watched the videos together 
without being able to pause, skip, fast forward, or repeat. The animated version was a voice-over PowerPoint that 
used the same narration, presented by the same teacher, that was used in the video teaching, making the presence 
versus absence of the teacher's face/body the only variable that differed between the two modes of teaching. The 
animations were played to students in the same way as the pre-recorded videos. 

(ii) In all conditions, students were asked to put away all other materials that might have served as distractions. 

(iii) Sample homogeneity was guaranteed. Firstly, the students had been randomly assigned to their classes 
three months prior to the study; therefore, the assignments to the groups were also random. Secondly, a t-test 
showed that the physics ability of the students in the two groups (as determined by their scores on the midterm 
physics exam conducted three weeks before the experiment) was similarly distributed. Thirdly, for the analysis of 
performance on each problem, only students who did not solve the corresponding pre-test problem entered the 
next stage of analysis. The binary outcome of whether the individual did or did not solve the one-item pre-test 
problem effectively filtered out results from students who were already adept at solving these problems, thus 
highlighting the teaching effect for students who initially lacked this skill. These measures minimise unobserved 
variation between students and groups. 

(iv) The post-test problems were reviewed by the team of 5 experts to make sure that they were directly related 
to the problems discussed in the teaching videos to ensure that what was tested was what the students should 
have learned. Students needed to use the same physics principles and problem-solving techniques taught in the 
teaching videos to solve the corresponding post-test problems. This enhanced the validity of the tests. 

(v) Given the unique nature of this study’s single-item, binary-scored assessment approach, traditional meth- 
ods and coefficients typically used for measuring test reliability and validity, such as Cronbach's alpha or inter-item 
correlations, are not applicable. However, the expert review of the problems, the source of the problems (national 
college exams), and the match of the failure rates for the pre- and post-tests indicate a good level of reliability. 


Limitations 


While this study provides valuable insight into the effectiveness of different video teaching formats in teach- 
ing physics problem-solving, it is important to acknowledge several limitations that may impact the general ap- 
plicability and interpretation of the results. 

Firstly, the study utilised only one teacher. This may introduce bias and limit the general applicability of the 
results to different teaching contexts and teaching styles. Future research should involve multiple teachers or adopt 
a cross-over design to minimise teacher-specific effects. 

Secondly, the experimental design included only one problem for each difficulty level. While this provided 
insight into the effectiveness of video teaching formats for specific problems, the results’ general applicability to 
a broader range of problem types and contexts may be limited. Including a more diverse set of problems would 
enhance the validity and applicability of the findings. 

Another limitation is the exclusive focus on upper secondary school students as participants. The results may 
not extend to other populations, such as college students or middle school learners. Examining participants from 
different age groups and educational backgrounds would provide a more comprehensive understanding of the 
effectiveness of different video teaching formats across diverse learner populations. 

Lastly, the study design did not incorporate long-term retention assessments. Assessing the durability of learn- 
ing outcomes and the sustained effectiveness of video teaching formats over time is essential for understanding 
their long-term impact on students’ problem-solving abilities. 
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Significance for Future Work 


While these limitations are important, this study yields valuable insight into the effectiveness of different video 
teaching formats for physics problem-solving. Future research should address these limitations by incorporating 
different teachers, a wider range of problem types, and diverse learner populations to further our understanding 
of effective teaching practice in physics education. Physics problem-solving requires high cognitive ability, logi- 
cal reasoning skills, and problem-solving techniques (Byun & Lee, 2014; Frensch & Funke, 2014; Park & Lee, 2004). 
Therefore, it is argued that the findings of this study have the potential to catalyse future work on the correlation 
between the effectiveness of video-based teaching and the difficulty level of the content beyond physics problem- 
solving, offering a template for educators worldwide to tailor video-based teaching effectively across varying 
educational contexts and cultural backgrounds. Specifically, investigating the effectiveness of video formats 
across various subjects that impose a high cognitive load on learners would provide valuable insights for teaching 
in different domains. Examining the applicability of the findings in subjects such as mathematics or other STEM 
disciplines would contribute to a broader understanding of effective teaching practices. Additionally, future stud- 
ies could investigate approaches for teaching complex topics that require high cognitive engagement, regardless 
of the specific subject area. By examining the transferability of our findings to different subjects and educational 
contexts, researchers can provide valuable guidance to educators regarding the selection of appropriate teaching 
methods to enhance learning outcomes and problem-solving abilities across a wide range of academic domains. 


Conclusions and Implications 


This study explored the effectiveness of pre-recorded video and narrated animation as teaching mediums in 
the context of upper secondary school physics problem-solving. Employing a controlled experimental design with 
361 students, the effectiveness of these video formats across varying levels of problem difficulty was compared 
systematically. The results revealed no significant differences in teaching effectiveness between pre-recorded video, 
which included the teacher's face, and narrated animation, which did not include the teacher's face, across any 
of the levels of content complexity. Unlike teaching factual knowledge, scientific concepts, and straightforward 
procedures, physics problem-solving necessitates complex reasoning, entails a high cognitive load, and demands 
high analytical skills. The results suggest that learning complex physics problem-solving, despite its high cogni- 
tive demands, does not influence the balance between the social cue benefits and the cognitive load drawbacks 
of including the teacher's face in educational videos. Furthermore, this equilibrium is unaffected by the teaching 
content's difficulty level. 

These findings suggest that educators may choose between pre-recorded and narrated animation based on 
factors other than the presence of the teacher's face, such as resource availability or personal preference, without 
compromising learning outcomes. For future research, it is suggested that the long-term retention of knowledge 
generated through these teaching formats might be examined and whether incorporating interactive elements 
could yield different results might be explored. Additionally, investigations should continue to assess the impact 
of teacher visibility in educational videos, considering diverse learning styles, learner demographics, and task 
complexity in conjunction with individual student differences such as prior knowledge. 
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Appendix 
Detailed Experimental Design 
Recorded Video Narrated Animation 
(N=154, 3 classes) (N=207, 4 classes) 
Worked out Worked out 
Problem 1 Failed Problem1 Failed 
N=95 Problem 1 N=141 Problem 1 
N=59 N=66 
Visualize the distribution of Mid-term physics exam scores 
100 for those who failed Problem 1 in each group 
2 
-) 
oO 
7) 
E 
GS 
* 
uu 
& 
[= 
8 
= 
= 
Video Animation 
Verify the invariance of the mid-term physics exam scores between two groups 
/ Calculate the percentage of correct answer to Problem \ 
% E 4009 1post for each group 
ag 
£06 5 
5 a = 80% 
22a 
85 o 60% 
as bed 
BLO 40% 
Loe 
_ 
S88 20% 
oe 
i) 
(om) 0 
Video Animation 
Example Problems 
Problem 1: 


A uniform cylinder with a mass of m is transported using a truck. To keep the cylinder in place, it is set be- 
tween two smooth inclines, as shown in the diagram. The inclines | and II are fixed on the truck and have angles 
of 30°and 60°, respectively. The acceleration due to gravity is g. When the truck travels at a constant speed along 
a straight road, what are the magnitudes of the forces exerted by the cylinder on inclines | and Il, represented by 
F1 and F2, respectively? 


586 


https://doi.org/10.33225/jbse/24.23.570 


Journal of Baltic Science Education, Vol. 23, No. 3, 2024 


RECORDED VIDEO VERSUS NARRATED ANIMATION IN TEAGHING PHYSICS PROBLEM- ISSN 1648-3898 | jerinty 
SOLVING: THE INFLUENGE OF PROBLEM DIFFICULTY LEVEL 
(PP. 550-569) ISSN 2538-71 38 /Online/ 


Problem 1 Post: 

As shown in the diagram, a balloon is in a static state in the wind, and the wind exerts a horizontal force on 
the balloon towards the right. The angle between the thin rope and the vertical direction is a, and the tension in 
the rope is T. What is the magnitude of the force exerted by the wind on the balloon? 
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